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Abstract 

This paper addresses the problem of sequence-based controller design for Networked Control Systems 
(NCS), where control inputs and measurements are transmitted over TCP- like network connections 
that are subject to stochastic packet losses and time-varying packet delays. At every time step, the 
controller sends a sequence of predicted control inputs to the actuator in addition to the current 
control input. In this sequence-based setup, we derive an optimal solution to the Linear Quadratic 
Gaussian (LQG) control problem and prove that the separation principle holds. Simulations 
demonstrate the improved performance of this optimal controller compared to other sequence-based 
approaches. 



1. INTRODUCTION 

In Networked Control Systems (NCS), components of a control loop are connected by one or more 
digital data networks. The applied data networks can be distinguished into two groups. There are 
real-time capable fieldbuses, such as Interbus, PROFIBUS, or Ether-CAT, which guarantee reliable 
data transmissions with deterministic latency on the one hand and general-purpose networks such 
as Ethernet-(TCP/UDP)/IP, WPAN (IEEE 802.15) or WLAN (IEEE 802.11) that have a stochastic 
transmission characteristic on the other hand. While fieldbuses have been the standard in industrial 
control systems for more than two decades, there is a trend towards applying general-purpose 
networks within the control loop for several reasons. Networks like Ethernet-TCP/IP are not 
only cheaper than the commercial fieldbuses but are also based on a wide-spread, nonproprietary, 
and standardized communication technology [I]. Furthermore, wireless networks are much more 
flexible than, e.g., the wired ring topology of Interbus, and allow for applications not realizable with 
fieldbuses such as mobile sensor networks [2] and freely moving objects [3] . 

However, general-purpose networks can be subject to highly time-varying transmission delays 
and to immense data losses. These effects can strongly degrade the performance of a system and 
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Figure 1: Considered setup: The sequence-based controller generates control input sequences XJ_ k that are transmitted 
to the actuator via a TCP-like network connection that provides acknowledgements for successfully transmitted data 
packets. The actuator holds the most recent control input sequence in a buffer and applies the time-corresponding 
entry u k of the sequence to the plant. The state x k of the plant is measured {y ) and sent via a network to the 
controller, which receives no, one, or even more than one measurement (Zk)- 



even destabilize the control loop [H G2 E] • Therefore, new control methods have been developed that 
take the stochastic network effects explicitly into account [TJ. 

Our approach belongs to a class of methods called sequence-based control, which is, depending 
on the author, also referred to as networked predictive control, packet-based control or receding 
horizon networked control [SJ El EH ITT] [T3?l [T3] . The main idea of this approach is that the controller 
sends data packets over the network to the actuator that contain not only the current control 
input, but also predicted control inputs for future time instants. The predicted future control 
inputs are stored in a buffer attached to the actuator so they can be applied in cases future data 
packets are delayed or lost. An assumption made by these methods is that the additional data 
sent over the network does not degrade the quality of the connection. For packet-based networks 
such as Ethernet-TCP/IP, this assumption usually holds since the process data needed for control 
applications is normally much smaller than the size of a data packet. 

Depending on the considered system, different approaches for design of sequence-based controllers 
have been proposed. One class of these approaches, as for example used in [T21 HH HHJ US], is 
based on a nominal feedback-controller that is designed for the nominal system, where the networks 
are replaced by transparent connections. Using the nominal controller, future control inputs are 
predicted and a sequence-based controller is synthesized. For linear systems, this approach can 
provide stability of the system, however, in the presence of packet losses, the resulting controller is 
in general not optimal even if the nominal controller is the result of an optimization method, such 
as LQG, H2, or Hoc,. 

Another line of sequence-based approaches evolves from Model Predictive Control (MPC) theory 
( |S1 EH E] ) ■ This is an intuitive connection since MPC-controllers already obtain an optimized 
sequence of predicted control inputs over a finite horizon. In standard MPC, only the first control 
input of this sequence is applied to the system and the rest is discarded. If instead the whole 
sequence is lumped into a data packet and sent to the actuator, a sequence-based controller is derived. 
However, like standard MPC, this approach is suboptimal since the real closed-loop optimization 
problem is approximated by the much easier open-loop optimization problem, which is suitable for 
systems that are far too complex to derive optimal solutions. 

In |13] . it has been shown, based on |l8j and |19j . that optimal sequence-based controllers in 



2 



the context of Linear Quadratic Gaussian (LQG) control can be derived for networks that use 
a so called TCP-like protocol. The term TCP-like^ characterizes a data network that provides 
instantaneous acknowledgments for successfully transmitted data packets |2U| . Yet, the approach 
in [T3] neglects the possibility of time delays by assuming that a data packet is either dropped by 
the network or transmitted immediately. In |21j . the approach was extended so that time delays 
could be incorporated into the sequence-based control design. However, the derived controller is not 
optimal since the authors artificially limit the information available to the controller for calculating 
control input sequences. 

In this paper, we present the optimal solution to the problem addressed in [13 and |21| also 
in the presence of time-varying transmission delays. In detail, we derive an optimal solution for 
the sequence-based LQG control problem for NCS with TCP-like network connections subject to 
stochastic packet losses and time-varying packet delays. The considered setup is depicted in fig. [T] 
The plant is assumed to be partially observable, discrete-time and linear and perturbed by additive 
process and measurement noise. The optimal control law is derived by first using state augmentation 
to formulate the original networked system as a nonnetworked Markovian Jump Linear System 
(MJLS). Then, stochastic dynamic programming is applied on the MJLS. In the derivation, we 
prove that the separation principle holds also in the sequence-based setup, what was assumed in 
former work but not formally proved. 

1.1. Notation 

Throughout the paper, random variables such as a are written in bold face letters, whereas 
deterministic quantities a are in normal lettering. Furthermore, the notation a ~ f(a) refers to a 
random variable a with probability density function /(a). A vector- valued quantity a is indicated 
by underlining the corresponding identifier and matrices are always referred to with bold face 
capital letters, e.g., A. The notation a& refers to the quantity a at time step k. For the set 
{x a ,x a+ i, . . . ,x b }, we use the abbreviated notation x a . b . The expectation operator is denoted by 
E{-} and the Moore- Penrose pseudoinverse of a matrix A by A^. The set of all natural numbers 
including zero is indicated with No and we use the symbol N>o for No\{0}. 

1.2. Outline 

The remainder of the paper is organized as follows. In the next section, the considered setup is 
introduced and the optimal control problem is formulated. The optimal control law is derived in 
sec. [3] and compared with the approaches from [T3] and |21| in a simulation of a double integrator 
system in sec. HJ A summary and an outlook on future work concludes the paper. 



1 A TCP-like network connection is only an approximation of a realistic Ethernet-TCP/IP network since it is 
assumed that the acknowledgements are not subject to time delays. However, the analysis of NCS with TCP-like 
connections gives insights into the more complex problem of controlling systems over real TCP connections or even 
over networks that do not provide any acknowledgments, such as networks using a UDP protocol. In particular, the 
TCP-like case constitutes an upper performance bound for NCS with realistic TCP or UDP network connections. 
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2. SYSTEM SETUP & PROBLEM FORMULATION 



We consider the system setup depicted in fig. [TJ The partially observable plant and the sensor 
evolve according to the stochastic, discrete-time, linear equations 

x k+ i = Az fc + Bu k + w k , (1) 
v k = Cx k + v k , (2) 

where x k G M n denotes the plant state at time step k, the vector u k G M. m the control input applied 
by the actuator, and y k G the measured output. The matrices A G W nxn , B G M. nxm , and 
C G IR 9 *™ are known and we assume that (A, B) is controllable and (A, C) is observable. The terms 
w k G M n and v k G represent mutually independent, stationary, zero-mean, discrete-time, white 
noise processes with Gaussian probability distribution that are independent of network-induced 
effects. The initial condition of the state x is independent of the other random variables and has a 
Gaussian distribution. We define 



x 



E {xq} and P = E {(^ - x ) (x - x ) T } 



The data connections between controller and actuator (CA-link) and between sensor and controller 
(SC-link) are provided by data networks that are subject to time-varying delays and stochastic 
packet losses. By interpreting lost transmissions as transmissions with infinite time delay, we unify 
the description of both effects by only considering time- varying but possibly unbounded time delays. 
The time delays are described by random processes T k A , T k c G No that specify how many time steps 
a data packet will be delayed if sent at time step k. Throughout the paper, we assume that T k A and 
T k c are white stationary processes and that their discrete probability density functions f SC (T k c ) 
and f CA (T k A ) are known. Additionally, it is assumed that the components of the control loop 
are time-triggered, time-synchronized and have identical cycle times. Furthermore, the employed 
network is capable of transmitting large time stamped data packets and uses a TCP-like protocol, 
i.e., acknowledgements are provided within the same time step when a packet was successfully 
transmitted. 

Every time step, the measurement y k is sent to the controller. Due to transmission delays and 
packet losses, it is possible that the controller receives no, one, or even more than one measurement. 
The set of received measurements at time step k G N>o is defined as the set Z k according to 

Z k = {y m ■ m G {0, 1, • • • ,k} ,m + rjp = k) . (3) 

After processing Z k) the sequence-based controller generates a control input sequence U_ k that 
is sent over the network to the actuator. Entries of that sequence are denoted by u k + m \ k with 
m G {0, 1, N} and G No- The index specifies that the control input is intended to be applied at 
time step k + m and was generated at time step k. This way, a sequence of length N + 1 generated 
at time step k is described by 



Hi 



T T T 
^k\k ^k+l\k ■ ■ ■ ^-k+N\k 



(4) 



Attached to the actuator is a buffer, in which the actuator stores the sequence with the most recent 
information among all received sequences, i.e., the sequence that was generated last (according to 
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the time stamps). If a sequence arrives out of order, the actuator does not change the content of 
the buffer and the received sequence is discarded. After updating the buffer, the actuator applies 
the control input of the buffered sequence that corresponds to the current time step. Since we do 
not assume that the time delays are bounded, it may happen that the buffer runs empty. In this 
case the controller applies a time-invariant default control input u d . Furthermore, the actuator 
initializes the buffer at time step k = with a sequence of default control inputs. The described 
actuator procedure can formally be summarized by 

Uk = Uk\k-e k , (5) 
6 k = min (In G N : m + r^ A = k-n,me N | U {N + 1}) , (6) 

y*\k-N-i = u d ■ ( 7 ) 



Remark 1 The random variable k can be interpreted as the age of the sequence buffered in the 
actuator, i.e., the difference between time step of generation and actual time step. If no appropriate 
control input is buffered in the actuator, Q k is set to N + 1 and the default control input u d is applied 
according to Q. 

As we consider a TCP-like protocol, the controller can always infer which control input has been 
applied to the plant. Therefore, at time step k, the controller has access to the past realizations of 
Ok- Based on the previous passages, we can summarize the information available to the controller 
at time step k by the information set X k according to 



0; FV Zl:k, Ux):k-li ^Orfc-l 



} • (8) 



At every time step k, the controller maps the available information to a control input sequence XJ_ k . 
Denoting the mapping function at time step k with \i k , the control law is given by the set of all [ik 
with k € {0, 1, . . . , K — 1}, where K is the terminal time. A control law is called admissible if 

U k = fM k (X k ) , V ke{0,l,...,K-l} (9) 

holds. In this paper, we are interested in finding an admissible control law that minimizes the 
cumulated linear quadratic cost function 

C<f = E | Cat + £ C h \u j0:K _ 1 ,x ,P ) , (10) 

I fc=0 J 

with stage cost 

C K = XkQ k x k , (11) 

Cfc = xlQkXk + Mfc RfcMfc , (12) 

where K G N>o is the terminal time step, Qk is positive semidefinite, and Rfc is positive definite. 

Summarizing the optimal control problem, we seek to find an admissible control law according 
to |)]) that minimizes the cost ( |10| ) - ( |12j ) subject to the system dynamics ([TJ, the measurement 
equations §2§, Q, and the actuator logic (pj - Q. 
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3. DERIVATION OF THE OPTIMAL CONTROLLER 



In order to derive the optimal controller, we model the system as a MJLS in sec. 3.1 Based on 



this model, we derive the optimal controller via stochastic dynamic programming in sec. 3.2 



3.1. System Modeling 

According to ^ and Q, the control input applied by the actuator at time step k is given by 

A 



ILk — U.k\k-e k > Mfc|jfe-jv-i 



u 



(13) 



where Ok is described by Q. As mentioned in remark [TJ the random variable Ok can be interpreted 
as the age of the sequence buffered in the actuator. Furthermore, it has been shown in |15| and [2"2"] 
that Ok can be described as state of a Markov chain with transition matrix T according to 
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(14) 



with 



Pij = Prob [0 k+1 = j\O k = i] 



N + l . 



The elements of T in the upper right triangle are zero as Ok can only increase by one per time step. 
The remaining entries can be calculated by 



P(i-m) 
Pij 



i - 



t-1 



s=0 



for i e [1,2,- ■■ ,N + 1] 



for j < i , where i,j S [0, 1, • • • ,N + 1] 



where qi is the probability that a sequence is delayed for i E No time steps. The q^s can directly be 
derived from f CA (T k A ). For a detailed derivation of the transition matrix we refer to |15j . 

With Ok, we can describe which sequence is buffered at time step k. To summarize all control 
inputs of sequences that could be buffered, we introduce the vector 
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with rj k G M. d and d = m + m ■ J2iLi ■ ■ ■//. 

all control inputs of the former sent sequences U_k-ii ' 



The vector r\ h contains the default control input u d and 
> LLk-N t na t s t m could be applied by the 
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Figure 2: Representation of control input sequences lZk-31 ■ ■ ■ , CZjt , whereas control inputs corresponding to the same 
time step are vertically aligned. The default control u d is added to the end of every sequence. The control inputs that 
could be applied by the actuator at time step k are marked by the green doted rectangle and the control inputs that 
are part of r\ h are marked by the red dashed rectangle. 



actuator either in the current time step or in the future. This is illustrated in fig. [2j where the 
relevant control input sequences are depicted for the case of N = 2. Therefore, the control input 
applied by the actuator at time step k is either part of r\ k or of Uk- 

Combining r] k and 0^, the following state space model of network and actuator can be derived 



with 
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F % + GU k , 



(16) 
(17) 



G 



I 




Thereby, the terms denote matrices with all elements equal to zero and I the identity matrix, each 
of appropriate dimension. The expression 5(e fci i) is the Kronecker delta, which is defined as 



1 if 6 k = i 
if k ^i 



Defining the augmented state 



V, 



(18) 
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and combining ([!]), fll~6| ), and (17), it holds that 



A B H fc 
F 



A fc | fc + B k U k + w fc 



B • Jfc 
G 



R k + 



(19) 
(20) 



The derived model (20) links the generated control input sequences with the state of the plant 
and the buffered control inputs. Due to the stochastic parameter 6 k , the augmented system is a 
Markovian Jump Linear System. 

3. 2. Calculation of the Control Law 

In the following, we will use the MJLS model (20) to derive the optimal control law by dynamic 
programming. 



Remark 2 In literature, there are several solutions to the LQG control problem of MJLS, e.g., 
However, these solutions cannot be applied directly since the mode, i.e., 6 k is supposed to be known 
at time step k. In our problem k is only known with a delay of one time step. Another difference 
is that the weighting matrices Q k and Ti k are stochastic. 



We define the minimal expected cost-to-go J k by 

J% = minE {C k + Jt +1 \T k ) , 

Jr = ^>{Ck\Zk} ■ 
According to dynamic programming theory [21] it holds, that 



(21) 
(22) 



JZ = min C£ 



! 



where Cq denotes the expected cumulated cost (10). Before we can use (21), we have to express 
the stage cost (11 ) - (12) in terms of the augmented system state £ fc . It holds, that 



C K = xJcQkXk = i K 



Qk 




C k — x£ QkX k + Mfc R-fcMfc 



x k Q k x k + (H fc r/ fc + J fc t/ fc ) R fc (H fc r/ fc + J k U k ) 



Qa 



HjR fc H fc 



x k 



T T 

+ H-k Jfc R-fcJfclZfc 



i T k Qki k +nI^kLLk , 



where we used that E |h^Rj. • 3 k \ = and introduced the definitions 



Qa 



Qk 




Qfc 



Qfc 
HlR fe Hfc 



Rfc 



Jfc RfcJfc 



(23) 
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Starting at time step K, the minimal expected cost-to-go are given by (22) 

J* K = E{|J Q K ZJL K } = E {e K K K i K \TK} 

with K.k = Qk ■ 



(24) 



With (21 ), it holds for the minimal expected cost-to-go at time step K — 1 
E {f K ^K-ii K _ x + UI-^k-iILk-i + J*k\Zk-x} 



J 



K-l = mm 
Jl 



E {il^QK-iij^^K-i} + min [E [uI^k-iUk-iVk-i] + E {^K^X^x} 
E {iLiQ^-i^-il^'-i} + ^ Wk-i E {Rjf-ilXjc-i}^-! 

i±-if-l . 



(25) 



+ 



e{(a 

E {S-i (Q^-i + ALiKifAx-i) ^JX^-i} 



+ min 



t^-i E jRfir-i + B k _ 1 ~KkRk-i\Tk-i}u k _ 1 



+ 2-E{g._ 1 |X A -_ 1 }E{A^_ 1 K x B if _ 1 |X /( _ 1 }^_ 1 ] + b{w^_ 1 K k w ic _ 1 \X K - 1 } , (26) 

where we used that E{E{g(£ )|Xfc-fi}|Xfc} = E{g(£ )\Ik} f° r ari Y function #(•). Furthermore, we 
used the fact that if Tk-i is given, then $ K1 is conditionally independent of Ok-i an d, therefore, 
of A^_i,Bx-i, and K A . Differentiation of (26) with respect to U_k—i an d setting equal to zero 
yields 



U 



K-l 



(E {Rjf_i + B^_ 1 K^B^_ 1 |X A _ 1 }) t x E {b^KkA^-iIXk-i} E {^JX^-i} 

(27) 



Remark 3 In (27), u>e /wzne used i/ie Moore-Penrose pseudoinverse instead of the regular inverse 
as the expression M = E |R; 



W\ -i + B^-_ 1 KA'Bii'_i| 



l|Xr-_i| is in general not regular but positive 
semidefinite. This results from two facts: 1) if the network has a latency such that a control input 
sequence cannot arrive before the first m E N>o time steps, i.e., qo = ■ ■ ■ = q m -\ = with qi as 



defined in sec. 3.1, then the first m — 1 control inputs of each sequence will never be applied and 
2) the last N control input sequences ILk-N-.k-x contain control inputs such as Uk+i\K—\ that 
are supposed to be applied after the terminal time K. Therefore, the minimization problem is not 
well-defined. One way to cope with this problem is to exclude the corresponding control inputs 
from the system equations. For case 1 ) this can be easily done by reducing the vector r/ k by the 
corresponding control inputs and adjusting the system matrices. In case 2) we have to gradually 
increase the dimension ofrj and the system matrices from time step K to K — N . Then, the system 
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has reached the dimension of the model derived above and the dimensions stay constant. Another 
way to solve this issue, is to use the Moore-Penrose pseudoinverse, which yields the same results 
since the kernel of M^M is equal to the subspace corresponding to the dimensions of the undefined 
entries ofU k . In this paper, we choose the latter description to obtain a concise expression for the 
optimal controller, which is straightforward to implement. 



Using (27) in (|26|) gives 

T* 

J K-1 



+ E{(!£_ i -E{g_ 1 |J Jf _ 1 })Ptf-i x ^ K l -E{i K l \l K ^})\l K ^} , 



with 



Kjf-i = e{q^_ 1 +A£_ 1 K x A a _ 1 |X a -_ 1 } 

P.fc-1 = E |a^-_ 1 KxB^_i|Xx-i} 

x (e{r a _! + B T K _ 1 K K B K „ 1 \l K ^ l }) ] x E^I^KkAk-iIIk-i} 

Considering one more time step, the minimal expected cost-to- go at time step K — 2 can be calculated 
by 



K-l 



(28) 
(29) 



T* 
■J l< 



K-2 



■ mm 

U K -2 



IK-2 



+ ■UjC-2 R -K-2Uk-2 + Jk-i\£k-2,ILk-2) 



(30) 



\Zk-2 \ + min U K _ 2 E \&k-2\1k-2 } U-K-2 

U-K-2 

+ E {e K1 K K ^ K1 \l K ^U K ^}} + E - E {g.jZjr-i}) Pjr-i 

x fer-i - E fe-il J ^-i}) ^-2,^-2} +E{^-iK^-i|Xx- 2 } • (31) 

The term with Pk-i is excluded from the minimization since it is independent of U_k—2 what is 
justified in Lemma [T] at the end of this section. The structure of (31) and (25) is the same, besides 
two additional terms that are independent of IZo:fc-i- Therefore, minimization over U_k-2 wm l ea< i 
to a J k-2 °f the same structure, so that it follows by an inductive argument that 



with 



U k = - (E {R fc + BjK k+1 B k \l k }) j E {BjK k+l A k \l k } E {|JX*} 
Kfc = E |Qfc + A^Kfc+iAfclXfcj — E ^AjKdk+iB k \l k ^ 



(32) 



x 



(E {R fc + B^K fc+ iBfc|Xfc|^ E {B^Kfc+iAfclXfcj 



(33) 



With ( 32 ) and ( 33 ) , it follows for the minimal expected cost-to-go at time step k that 

K-l 



J* k =E {gKfc +1 |JJfc} + ]T E {wjKi^Wipi} + E {{iJ +1 ~ E {C+il^+i}) P m 

x fe +1 - E fe +a M)l Z *} • (34) 
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The expected values in (32) and (33) referring to the matrices can be calculated by explicitly 



conditioning on k -\ = j- This is possible since 6 k _\ is part of the information set T k and, therefore, 
known at time step k. It holds 



Hi 



N+l 

Pji (% + B J E {K k+1 \0 k = i} B, 

L i=0 



nt r 



N+l 



J2 Pj Mv{K k+ i\e k = i}A ] . 



li=0 



— LfcE |^ fc |Ife| 



•E{|JJ fe } 
(35) 
(36) 



with 



E{K k \6 k _ 1 =j} 



N+l 



Pji [Q\i + Aj E{K fc+ i|0 fc = i} A, 



Li=0 



N+l 



53p ii AjjE{K fc+1 |e fc = i}B| 



i=0 



£ Pi< (% + Bf E {K fc+ i|0 fc = i} B 



i=0 



7V+1 



^^BjE{K fc+1 |6> fc = i}A 



i=0 



(37) 



where the notation Xij, with i 6 No, refers to the matrix X (dependent of 6 k ), where k is set to i. 
The terms indicate the elements of transition matrix T from ( |14[ ) . 
The results derived above can be summarized as follows: 

Theorem 1 Consider the problem to find an admissible control law with given sequence length 
N according to (|9| that minimizes the cost (10) - (12) subject to the system dynamics ([!]), the 
measurement equations ^ and and the actuator logic (j^J) - ([T]) . Then, 

1. as in standard LQG control, the separation principle holds, i.e., the optimal control law at 
time step k can be separated into a) an estimator that calculates the conditional expectation 
E|^ fc |Xfc| and b) into an optimal state feedback controller that utilizes the state feedback 
matrix h k . 

2. the optimal control law is linear in the conditional expectation of the augmented state, i.e., 
U k = L fe E j^jZfcj. 

the optimal state feedback matrix ~L k can be calculated by (35), whereas the term E {JZ k+ i\6 k = i} 
is obtained by the recursion (37), which is evolving backwards in time, with initial condition 
E {Kk\0k-i 



3. 



i} = Q 



K ■ 



Since rj k in (18) is known at time step k, the conditional expectation E|^ fc |Xfc| reduces to 
E{x k \I k }, which is equal to the minimum mean squared error estimate of the state x k . In the 
literature, results on the minimum mean squared error estimator in presence of measurement 
delays and measurement losses are available, see e.g., |25| and [2S]- It was pointed out in J2S] that 
the optimal estimator is a time- varying Kalman filter that is extended by a buffer to store old 
measurements. The buffered measurements are needed to incorporate delayed measurements that 
are received after measurements with more recent information. This filter is of finite dimension (and, 
therefore, can be implemented) when the required memory of the buffer is finite. This is the case if 
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every measurement delay is either bounded or infinite^} In practice, however, it is computational 
inefficient to process measurements that have been delayed for a very long time and, therefore, the 
length of the buffer is treated design parameter. 

For similar reasons, we assumed in the problem formulation that the length N of the control 
input sequence is a given design parameter. Otherwise, if the minimization would be carried out 
over N, and we consider networks with data losses, then the resulting length of the control input 
sequence would be infinite. 

Lemma 1 The term 

is stochastically independent of the control sequence U_h-i- 
Proof. 



Consider the system (20) and the autonomous system, given by 



(38) 



that has same system matrix, initial conditions, noise realizations w 0:k _ 1 , Vo:fc-l> and network delay 
realizations tS|_ 1; T^. k _ l . Both systems evolve according to time- variant transformations, which 
are linear, so that it is possible to find matrices A, B, and C depending on the realizations of #o:fc— l 
with 



-k-i 



+ C 



w 



> 



A| + C 



T 



w 



fc-1 



It holds for the expected values 

E{! fc |X fe } = AE{gx fc }+B 



E{|jX fe }=AE{| |I fe } , 



where A and B are known since the information vector X k includes 0o:fc-l- Defining the estimation 
errors e k = | fc - E {| fc |X fc | and e k = | fc - E {tjZfc}, it holds that 

, T 



Cfc = A(| -E{| |25fe}) + c(teJ ••• wl^] 

Therefore, the errors are identical and since 

efc = e k = | fe - E {tjZfc} = | fc - E {tjz 0; fc, 9o±-i} 



2 This formulation is consistent with the one in |25| . where it is required for the finite optimal filter to exist that a 
measurement, that is not lost, will arrive within a maximal time. 
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it holds that the error is independent of U_h-i an d that 

E|efee fe |X fc -i,L[fe_i} = E|efce fe |z :fc-i,^0:fe-2} • (39) 

If 0fc_2 is given, then and 0^ are conditionally independent of Zk-i, ILk-2 an d LLk-i- It follows 
that 

E{P fc |J fc _i,t^_ 1 } = E{P fc |» J fc_ a } (40) 

and finally 

E je£Pfcefc|Zfc_i, IZft-i} = E |e^P fc e fc |z 0;fc , 0o:fc-i} 
concludes the proof. □ 
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Figure 3: Probability density functions of the arrival probability of data packets over time delays. 
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4. SIMULATIONS 



In this section, we compare the performance of the proposed optimal controller with the 
approaches presented in |13| and |21] by means of simulations with a double integrator. For 
simulation, the system parameters of ([!]) and ^ are chosen with 



1 1 
1 



B 



1 



and the weighting matrices of the cost function (10), the initial condition and the noise covariances 
are set to 



1 
1 



Q 



R = 

0.1 2 
0.1 2 





ioo~ 




0.5 2 


' 


x = 





, Po = 





0.5 2 







E 



{ v k v k} 



0.2" 



In the simulation, we use two different models of the network connections. The probability density 
functions of the delay distributions of both networks are depicted in fig. [3} Thereby, Network A 
has a better transmission quality than Network B as, first, the probability of a small time delay 
is significantly higher and, second, the loss probability, i.e. the probability of an infinite delay, is 
much smaller. We assume that the probability density function of the controller-actuator network is 
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Figure 4: Resulting averaged cumulated LQG cost for Network A and Network B for different lengths of the control 
input sequences. 



the same as the one of the sensor-controller network. To obtain the minimum mean squared error 



estimate of the state, i.e., the conditional expectation in (36), we employed the filter described in 
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|26j . The filter is chosen so that it can process measurements with a delay of more than 10 time 
steps and, therefore, yields the optimal state estimate. If the buffer of the actuator runs empty the 
actuator applies the default control input u d = 0. 

For each controller and network and for different length N of the control input sequence, we 
conduct 500 Monte Carlo simulation runs over 40 time steps and calculate the average of the 
cumulated cost ( 10 ). The results are shown in fig. [I] The proposed controller leads to the lowest cost 
for both networks. For Network A, the difference between the three controllers is only small. This 
results from the good transmission quality of the network, in particular, from the high probability 
that a data packet will arrive at the actuator without any delay. This is extremely beneficial for the 
approach in [13], where only undelayed packets can be used by the actuator, and for the approach 
in |21j . which is based on the approximation that the controller uses only the state information 
deemed available to the actuator (see |21| for more details). 

For Network B the approach of [21] is not depicted as it is not able to stabilize the system, i.e., 
leads to cost that are several magnitudes higher than the cost of the other controllers. Compared to 
Network A the cumulated cost of the proposed controller and the controller from [13| are higher, 
what results from the worse transmission quality, which unavoidably degrades the performance. 
However, for a sequence length of N > 1 the proposed approach leads to half of the cost compared 
to the approach of [p3] . 



5. CONCLUSIONS 

We presented an optimal solution to the sequence-based LQG control problem for NCS with 
TCP-like network connections. In contrast to former work, we were able to optimally consider 
time-varying packet delays in the sequence-based controller design. 

Future work will be concerned with a derivation of stability conditions for the proposed controller. 
It seems reasonable to assume that the stability region is larger than the one of the approaches in 
[T3] and [21], but this has still to be proved. Furthermore, we seek to include a simple cost model 
of the network into the cumulated cost function so that the controller also considers the power 
consumption of a transmission. This way, the minimization is also carried out over the length of 
the control input sequences and the controller may, e.g., decide not to send a sequence since the 
buffered sequence is still good enough. Finally, it is an aim to relax the assumptions of the TCP-like 
network connection in favor of realistic TCP connections with possibly delayed acknowledgements. 
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